BOSTON — Three years ago, Massachusetts enacted perhaps the boldest state health care experiment in American history, bringing near-universal coverage to the commonwealth with Paul Revere speed.

To make it happen, Democratic lawmakers and Gov. Mitt Romney, a Republican, made an expedient choice, deferring until another day any serious effort to control the state’s runaway health costs.

The day of reckoning has arrived. Threatened first by rapid early enrollment in its new subsidized insurance program and now by a withering economy, the state’s pioneering overhaul has entered a second, more challenging phase.

Thanks to new taxes and fees imposed last year, the health plan’s jittery finances have stabilized for the moment. But government and industry officials agree that the plan will not be sustainable over the next 5 to 10 years if they do not take significant steps to arrest the growth of health spending.

With Washington watching, the state’s leaders are again blazing new trails. Both Gov. Deval Patrick, Mr. Romney’s Democratic successor, and a high-level state commission have set out to revamp the way public and private insurers reimburse physicians and hospitals.

They want a new payment method that rewards prevention and the effective control of chronic disease, instead of the current system, which pays according to the quantity of care provided. By late spring, the commission is expected to recommend such a system to the legislature.

If Massachusetts becomes the first state to make this conversion, health policy experts argue that it would be as audacious an achievement as universal coverage. The state faces several hurdles, including securing federal permission to impose the changes on Medicaid, a shared state and federal program, and more unusually on Medicare, which is financed entirely by Washington.

Those who led the 2006 effort said it would not have been feasible to enact universal coverage if the legislation had required heavy cost controls. The very stakeholders who were coaxed into the tent — doctors, hospitals, insurers and consumer groups — would probably have been driven into opposition by efforts to reduce their revenues and constrain their medical practices, they said.

Now those stakeholders and the state government have a huge investment to protect. But the task of cost-cutting remains difficult in a state with a long tradition of heavy spending on health care. Massachusetts has more doctors per capita than any state, Boston is home to some of the country’s most expensive academic medical centers, and a new state law requires comprehensive benefits like prescription drug and mental health coverage.

Alan Sager, a professor of health policy at Boston University, has calculated that health spending per person in Massachusetts increased faster than the national average in seven of the last eight years. Furthermore, he said, the gap has grown exponentially, with Massachusetts now spending about a third more per person, up from 23 percent in 1980.

“Just as this may have been the easiest place to do coverage, it may be the most difficult place to do cost control,” said Jonathan Gruber, a health economist at the Massachusetts Institute of Technology.

But Mr. Patrick has shown signs of playing tough with the state’s hospitals and insurers. Responding in January to a series in The Boston Globe that exposed how the state’s most influential hospitals negotiate high reimbursement rates, Mr. Patrick announced that he would explore whether the state could regulate insurance premiums.

“Frankly, it’s very hard for the average consumer, or frankly the average governor, to understand how some of these companies can have the margins they do and the annual increases in premiums that they do,” Mr. Patrick said in an interview. “At some level, you’ve just got to say, ‘Look, that’s just not acceptable, and more to the point, it’s not sustainable.’ ”

The threat seems to have been heard. Insurers seeking to participate in the state’s subsidized insurance program, Commonwealth Care, recently submitted bids so low that officials announced last week that they would keep premiums flat in the coming year. That may provide cover for the program as the state seeks ways to fill a nearly $4 billion gap in its 2010 budget.

The state expects to spend $595 million more on its health insurance programs this year than in 2006, a 42 percent increase. But about 432,000 people have gained coverage, leaving only 2.6 percent of the population without insurance, according to a recent state survey. At only one-sixth the national average, that is by far the lowest rate in any state.

Massachusetts achieved its high coverage rates by mandating in its landmark law that almost every resident have health insurance, and that all but the smallest businesses make some contribution toward their employees’ costs. Those who do not enroll but are deemed able to afford insurance can be fined up to $1,068 in the 2009 tax year.

To make the mandated insurance affordable, the state subsidizes premiums for those earning up to three times the federal poverty level, or $66,150 for a family of four. Massachusetts already had a law requiring insurers to accept all applicants regardless of their health status.

Although nearly 60 percent of the newly insured are covered by public programs, Massachusetts also seems to be a rare state where the percentage of employers offering health benefits is actually growing. And the state government has realized substantial savings, worth about $250 million last year, from lower payments to hospitals for uncompensated care for the uninsured and underinsured.

In its first full year of operation, Commonwealth Care drew higher enrollment than anticipated, and the state found itself facing an inaugural budget gap. Mr. Patrick and the legislature filled it by assessing insurers and hospitals, raising the penalty on noncompliant businesses, increasing premiums and co-payments for consumers, and raising the state tobacco tax.

The fear was that such tree-shaking would become an annual ritual. But enrollment in the $820 million Commonwealth Care program peaked last May and then declined before hitting a plateau.

Some modest provisions to control costs were included in the original health care bill, including a merger of the small group and individual insurance markets and new spending on electronic record-keeping and hospital infection control.

More efforts were made last year in legislation that provided incentives for doctors to practice primary care, required uniform billing procedures among providers, toughened the state’s regulation of new hospital construction, and established the payment reform commission.

The commission is looking at various options, but all would do away with the fee-for-service system, which provides perverse incentives by paying physicians and hospitals for each patient visit. The changes under consideration include reimbursing for episodes of care rather than individual visits and bundling payments to groups of providers who would together take responsibility for a patient’s health.

Blue Cross and Blue Shield of Massachusetts, the state’s largest insurer, recently devised an innovative model that pays doctors a flat fee per patient, with adjustments for age, gender and health status, and then adds bonus payments for high standards of care.

Blue Cross officials say they believe that the new plans can cut the growth of premiums in half over five years and expect them to account for 15 percent of their business by June. “We’re very committed to this path because we feel it’s the only credible place to go,” said Cleve L. Killingsworth, the company’s chairman.

Some health policy experts argue that changes in payment practices will not be enough to slow the growth in spending, even when combined with other cost-cutting strategies. To truly change course, they say, the state and federal governments may need to place actual limits on health spending, which could lead to rationing of care.

“Really controlling costs requires just stopping spending,” said Stuart H. Altman, a professor of health policy at Brandeis University.

Because Massachusetts now requires its residents to be insured, it cannot fall back on the strategy used by other states in hard times — to simply remove people from the public insurance rolls by restricting eligibility.

“It forces us to look in the mirror and say, ‘What do we do about health care spending?’ ” said Jon M. Kingsdale, executive director of the agency that administers Commonwealth Care. “And the reason that’s so challenging is that it means limiting resources for people doing really good stuff.

“It’s not like the fat sits out here easily identified and you just slice it off. It’s marbled throughout the meat.”